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Abstract 



A constructive version of Hausdorff dimension is developed using constructive supergales, 
which are betting strategies that generalize the constructive supermartingales used in the theory 
of individual random sequences. This constructive dimension is used to assign every individual 
(infinite, binary) sequence S a dimension, which is a real number dim(S') in the interval [0, 1]. 
Sequences that are random (in the sense of Martin-L6f) have dimension 1, while sequences that 
are decidable, S^, or have dimension 0. It is shown that for every ASj-computable real 
number a in [0,1] there is a A§ sequence S such that dim(S') = a. 

A discrete version of constructive dimension is also developed using termgales, which are 
supergale-like functions that bet on the terminations of (finite, binary) strings as well as on their 
successive bits. This discrete dimension is used to assign each individual string w a dimension, 
which is a nonnegative real number dim(ui). The dimension of a sequence is shown to be the 
limit infimum of the dimensions of its prefixes. 

The Kolmogorov complexity of a string is proven to be the product of its length and its 
dimension. This gives a new characterization of algorithmic information and a new proof of 
Mayordomo's recent theorem stating that the dimension of a sequence is the limit infimum of 
the average Kolmogorov complexity of its first n bits. 

Every sequence that is random relative to any computable sequence of coin-toss biases that 
converge to a real number (3 in (0, 1) is shown to have dimension TL{(3), the binary entropy of [3. 

Keywords: algorithmic information, computability, constructive dimension, dimension, entropy, 
gales, Hausdorff dimension, Kolmogorov complexity, Kullback-Leibler divergence, martingales, ran- 
domness, supergales, termgales. 



1 Introduction 

One of the most dramatic achievements of the theory of computing was Martin-L6f 's 1966 use of 
constructive measure theory to give the first satisfactory definition of the randomness of individual 
infinite binary sequences p6f| . The search for such a definition had been a major object of early 
twentieth-century research on the foundations of probability, but a rigorous mathematical formu- 
lation had proven so elusive that the search had been all but abandoned more than two decades 
earlier. Martin-L6f 's definition says precisely which infinite binary sequences are random and which 
are not. The definition is probabilistically convincing in that it requires each random sequence to 
pass every algorithmically implementable statistical test of randomness. The definition is also ro- 
bust in that subsequent definitions by Schnorr [36, 37, 38], Levin [pQ l, Chaitin Q, Solovay [14], 
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and Shen' |4^, ^T|, using a variety of different approaches, all define exactly the same sequences to 
be random. It is noteworthy that all these approaches, like Martin-L6f 's, make essential use of the 
theory of computing. 

One useful characterization of random sequences is that they are those sequences that have 
maximal algorithmic information content. Specifically, if K(S[0..n — 1]) denotes the Kolmogorov 
complexity (algorithmic information content) of the first n bits of an infinite binary sequence S, 
then Levin |20| and Chaitin || have shown that S is random if and only if there is a constant c 
such that for all n, K(S[0..n — 1]) > n — c. Indeed Kolmogorov [17| developed what is now called 
C(x), the "plain Kolmogorov complexity," in order to formulate such a definition of randomness, 
and Martin-L6f, who was then visiting Kolmogorov, was motivated by this idea when he defined 
randomness. (The quantity C(x) was also developed independently by Solomonoff |2j| and Chaitin 
(|. H].) Martin-L6f |27j subsequently proved that C(x) cannot be used to characterize randomness, 
and Levin [ 20 1 and Chaitin || introduced a technical modification of C(x), now called K(x), 
the "Kolmogorov complexity," in order to prove the above characterization of random sequences. 
Schnorr [38] proved a similar characterization in terms of another variant, called the "monotone 
Kolmogorov complexity." 

One conclusion to be drawn from these characterizations is that the definition of random se- 
quences distinguishes those sequences that have maximal algorithmic information content from 
those that do not. It offers no quantitative classification of the sequences that have less than 
maximal algorithmic information content. From a technical point of view, this aspect of the defini- 
tion arises from its use of constructive measure, which is an algorithmic effectivization of classical 
Lebesgue measure. Specifically, an infinite binary sequence S is random if the singleton set {S} 
does not have constructive measure 0, and is nonrandom if {S} does have constructive measure 0. 
Neither Lebesgue measure nor constructive measure offers quantitative distinctions among measure 
sets. 

In 1919, Hausdorff [ 13 1 augmented classical Lebesgue measure theory with a theory of dimension. 
This theory assigns to every subset X of a given metric space a real number dimn^X), which is 
now called the Hausdorff dimension of X. In this paper we are interested in the case where the 
metric space is the Cantor space C, consisting of all infinite binary sequences. In this case, the 
Hausdorff dimension of a set ICC (which is defined precisely in section 3 below) is a real number 
dimn(X) G [0,1]. The Hausdorff dimension is monotone, with dimjj(0) = and dimn(C) = 1. 
Moreover, if dimn(^) < dimn(C), then X is a measure subset of C. Hausdorff dimension thus 
offers a quantitative classification of measure sets. Moreover, Ryabko |}^, [34], 35] Staiger [46, 47], 
and Cai and Hartmanis Q have all proven results establishing quantitative relationships between 
Hausdorff dimension and Kolmogorov complexity. 

Just as Hausdorff |l3j augmented Lebesgue measure with a theory of dimension, this paper 
augments the theory of individual random sequences with a theory of the dimensions of individual 
sequences. Specifically, we develop a constructive version of Hausdorff dimension and use this 
to assign every sequence S G C a dimension dim(S*) G [0,1]. Sequences that are random have 
dimension 1, while sequences that are decidable, S^ 1 , or n^. have dimension 0. For every real 
number a G [0, 1] there is a sequence S such that dim(S') = a. Moreover, if a is A^-computable, 
then there is a sequence S such that dim(S') = a. (This generalizes the well-known existence 
of sequences that are random.) 

Our development of constructive dimension is based on supergales, which are natural generaliza- 
tions of the constructive supermartingales used by Schnorr (3^, [}?], 38 1 to characterize randomness. 
In a recent paper [25| we have shown that supergales can be used to characterize the classical 
Hausdorff dimension, and that resource-bounded supergales can be used to define dimension in 
complexity classes. In the present paper we use constructive (lower semicomputable) supergales 
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to develop constructive dimension. The dimension of a sequence S E C is then the constructive 
dimension of the singleton set {S}. Constructive dimension differs markedly from both classical 



Hausdorff dimension and the resource-bounded dimension developed in [25], primarily due to the 
existence of supergales that are optimal. These optimal supergales are analogous to universal tests 
of randomness in the theory of random sequences. 

Supergales, like supermartingales, are strategies for betting on the successive bits of infinite 
binary sequences. In order to define the dimensions of individual strings w E {0, 1}*, we introduce 
termgales, which are super gale-like functions that bet on the terminations of strings as well as on 
their successive bits. Using termgales, we assign each binary string w a dimension dim(u>), which 
is a nonnegative real number. We show that for every sequence S E C, 



dim(5 l ) = liminf dim(S'[0..n — 1] 



1.11 



We use dimension to prove a new characterization of Kolmogorov complexity. Specifically, we 
show that there is a constant c such that for all w E {0, 1}*, 



K(w) — \w\ dim(u)) 



< c. 



;i.2) 



That is, the Kolmogorov complexity of a string is (to within a constant additive term) the product 
of the string's length and its dimension. This characterization of Kolmogorov complexity in terms 
of a constructivized, discretized version of Hausdorff 's 1919 theory of dimension is reminiscent of 
(and technically related to) the well-known characterization by Levin 2C, ^lj and Chaitin Q of 
Kolmogorov complexity in terms of constructivized discrete probability, i.e., the fact that there is 
a constant d E N such that for all w E {0, 1}*, 



K(w) — log 



1 



m [wj 



<c', 



where m is an optimal constructive subprobability measure on {0, 1}*. 

Taken together, ( |1.1| ) and (L2) provide a new proof of Mayordomo's recent theorem 
that for every sequence S E C, 



dim(S) 



lim inf 

n— >oo 



K(S[0..n-l}) 



;i.3) 



stating 



(1.4) 



Facts (|1.2|) and (|1.4[) justify the intuition that the dimension of a string or sequence is a measure 
of its algorithmic information density. 

We also investigate the dimensions of sequences that are random relative to computable se- 
quences of convergent coin-toss biases. Specifically, let (3 = (/3o, Pi, Pi, ■ ■ ■ ) be any computable 
sequence of real numbers Pi E [0, 1] that converge to a real number p E (0, 1) (which must therefore 
be Aj-computable). We show that if R is any sequence in C that is random with respect to P (i.e., 
a random outcome of a random experiment in which for each i, independently of all other j, the i th 
bit of R is decided by tossing a 0/1-valued coin whose probability of 1 is Pi), then the dimension 
of R is H(P), the binary Shannon entropy of p. 

We defer discussion of some significant related work until late in the paper, where more context 
is available. Specifically, results by Schnorr [37, gjj, Ryabko [32, 33, [M, pq ], Staiger [46, [47], |4q], and 
Cai and Hartmanis || that relate martingales, supermartingales, and Kolmogorov complexity to 
Hausdorff dimension are discussed at the end of section 6. Classical work by Besicovitch [jlj], Good 
[12], and Eggleston [||] relating limiting frequencies and Shannon entropy to Hausdorff dimension 
is described briefly in section 7. 
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2 Preliminaries 



We use the set Z of integers, the set Z + of (strictly) positive integers, the set N of natural numbers 
(i.e., nonnegative integers), the set Q of rational numbers, the set R of real numbers, and the set 
[0, oo) of nonnegative reals. 

A string is a finite, binary string w £ {0, 1}*. We write \w\ for the length of a string w and A 
for the empty string. For i,j G {0, ... ,\w\ — 1}, we write for the string consisting of the 

i th through the j th bits of w and w[i] for the i th bit of w. Note that the th bit w[0] is the 

leftmost bit of w and that = A if i > j. A sequence is an infinite, binary sequence. If S is 

a sequence and i, j G N, then the notations S[i..j] and S[i] are defined exactly as for strings. We 
work in the Cantor space C consisting of all sequences. A string w G {0, 1}* is a prefix of a sequence 
SeC, and we write w C S, if if?[0..[itj| — 1] = w. The cylinder generated by a string it; £ {0, 1}* is 
C„, = {SG C|u> C 5}. Note that C A = C. 

We also make passing references to ITj* , A[>, and sets of sequences. These refer to the arith- 
metical (i.e., effective Borel) hierarchy of sets of sequences and are not central to our development. 
The interested reader is referred to or for discussion of this hierarchy. 

The support of a sequence S G C is 

supp(S) = {n G N[5[n] = 1}. 

The arithmetical hierarchy of sequences is defined from the arithmetical hierarchy of subsets of N 
using the support function. Thus, for example, a sequence S G C is computably enumerable, and 
we write S G if supp(S) is a (i.e., computably enumerable) subset of N. Similarly, S £ Iq if 
N — supp(S') is computably enumerable, and S G A° if supp(S') is decidable relative to the halting 
oracle. 

If S, T G C, then S is 1-reducible to T, and we write S <i T, if there is a computable, one- 
to-one function / : N — * N such that for all n G N, S'fn] = T[f(n)]. The sequences 5 1 and T are 
1-equivalent, and we write S 1 =i T, if S <i T and T <i S 1 . 

Definition. A subprobability measure on {0, 1}* is a function p : {0, 1}* — > [0, 1] such that 

(ue{o,i}* 

A probability measure on {0, 1}* is a subprobability measure on {0, 1}* that satisfies ( |2.lD with 
equality. A subprobability supermeasure on the Cantor space C is a function : {0, 1}* — > [0, 1] 
such that 

f(A) < 1 (2.2) 

and for all u; G {0, 1}*, 

i/(tu) > v(w0) +u(wl). (2.3) 

A subprobability measure on C is a subprobability supermeasure on C that satisfies ( fE3| ) with 
equality for all u; G {0, 1}*, and a probability measure on C is a subprobability measure on C that 
satisfies ( p.2[ ) with equality. Intuitively, if ^ is a probability measure on C and w G {0,1}*, then 
is the probability that idCS when the sequence S G C is "chosen according to the probability 
measure u." 
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A bias is a real number f3 G [0, 1]. Intuitively, if we toss a 0/1-valued coin with bias /3, then (5 
is the probability of the outcome 1. A bias sequence is a sequence j3 = (/?o, fti, P2, ■ ■ ■) of biases. If 
/? is a bias sequence, then the f3- coin-toss probability measure is the probability [fl on C defined by 

M-i 

a*>) = n ( 2 - 4 ) 

i=0 

where $(10) = (2/3j — l)io[i] + (1 — i.e., /3j(u)) = if then $ else 1 — That is, pP is the 
probability that S G C w when S G C is chosen according to a random experiment in which for 
each i, independently of all other j, the i th bit of S is decided by tossing a 0/1-valued coin whose 
probability of 1 is In the case where the biases (3i are all the same, i.e., (3 = (/?,/?, /3, ... ) for 
some /3 £ [0, 1], we write /u' 3 for yfi , and ( [2.4D simplifies to 

/(w) = (l - / 3)#(0 > «) / 3#(i^) j ( 2 . 5 ) 

where #(6, u>) is the number of times the bit b appears in the string w. The uniform probability 
measure on C is the probability measure [i = p^, for which (|2.5D simplifies to 

ix(w) = 2-1^1 (2.6) 

for all w G {0,1}*. 

We use several conditions involving the computability of real numbers and real- valued functions 
in this paper. 

Definition. Let / : D — > R, where D is some discrete domain such as N, {0, 1}*,N x {0, 1}*, etc. 

1. / is computable if there is a computable function / :DxN->Q such that for all (x, r) G D x N, 
|/(x,r)-/(x)| <2" r . 

2. / is lower semicomputable if there is a computable function / :DxN->Q such that 

(a) for all (x,t) G £> x N, /(x,t) < /(x, t + 1) < /(x), and 

(b) for all x G D, lim^oo f(x,t) = f(x). 

3. / is Ag- computable if / is computable relative to the halting oracle. 
The following facts are well known and easy to verify. 

(i) Computability implies lower semicomputability, lower semicomputability implies A^-computability, 
and the converses of these statements do not hold. 

(ii) A function / : D — » R is computable if and only if the functions / and — / are both lower 
semicomput able . 

(iii) A function / : D — > 1R is lower semicomputable if and only if its lower graph 

Graph" (/) = {(x,s) G D x Q|a < f(x)} 
is computably enumerable. 

A sequence a = (ao,ai,a2, • • • ) of real numbers is computable if the associated function f$ : 
N — > R, defined by f${i) = cti for all i G N, is computable. A real number a is computable if the 
sequence a = (a, a, a, . . . ) is computable. Note that if f3 is a computable bias sequence, then /i' 3 is 
a computable probability measure on C. 
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Definition. A subprobability measure on {0, 1}* or a subprobability supermeasure on C is con- 
structive if it is lower semicomputable. 



Definition. If T is a class of functions from {0, 1}* into [0, oo), then an optimal element of T is a 
function g G T such that for every / G J- there is a real constant a > such that for all w G {0, 1}*, 
g(w) > af(w). 

The following theorem is one of the cornerstones of algorithmic information theory. 
Theorem 2.1. (Levin g]) 

1. There is an optimal constructive subprobability measure m on {0,1}*. 

2. There is an optimal constructive subprobability supermeasure M on C. 
Throughout this paper we fix m and M as in Theorem |2.1| . 

The reader is referred to the text by Li and Vitanyi |22[ for the definition and basic proper- 
ties of the Kolmogorov complexity K(w), defined for strings w G {0,1}*. The main property of 
Kolmogorov complexity that we use here is the following theorem, which is another cornerstone of 
algorithmic information theory. 



Theorem 2.2. (Levin |20|, glj, Chaitin g) There is a constant c G N such that for all w G {0, 1}* ; 

1 



K(w) — log 



m(w) 



< c. 



Given a set A C {0, 1}* and n G N, we use the abbreviations A =n = A n {0, l} n and A< n = 
An{0,l}- n . A prefix set is a set A C {0, 1}* such that no element of A is a prefix of another 
element of A. 

Let A be a /c-fold product of intervals, each of which is (0,1) or [0,1]. If g : X — ► M and 
a = (ai, . . . , a*;) G A, then we sometimes use u g(a)" as an abbreviation for the random variable 
f : {0, 1} -> R defined by = ^(ai,.. . ,a fc ) and f(0) = g(l-a u ... ,l-a k ). If /3 G [0,1], then 
we also use (5 as an abbreviation for the probability measure p on {0, 1} in which p(l) = (3. Thus, 
for example, 

E/3#(a) = fig{otx, ... , a k ) + (1 - /3)^(1 - ai, . . . , 1 - a k ) 

is the expected value of "the random variable <?(a)" with respect to "the probability measure /?." 
In particular, Shannon's binary entropy function Ti : [0, 1] — > [0, 1] is defined by 

W(/3)=E /3 logi, 

with the proviso that Olog jjj = so that 7i is continuous on [0, 1]. Similarly, the binary Kullback- 
Leibler divergence function D : [0, 1] x (0, 1) — > M is defined by 

D{fl || a) =E /3 log^, 
a 

with the proviso that Olog | = so that Z) is continuous on [0, 1] x (0, 1). It is well-known that 
D(f3 [| a) > 0, with equality if and only if /3 = a. See the text by Cover and Thomas || for further 
discussion of H(f3) and D{j3 \\ a). 

Falconer |L0| provides a good overview of Hausdorff dimension. 
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3 Gales and Constructive Dimension 

In this section we define gales and supergales and use these to define classical and constructive 
Hausdorff dimensions in the Cantor space C. Our definitions are slightly more general than those 
in |2f| because here we need to define gales and supergales relative to an arbitrary (not necessarily 
uniform) probability measure on C. 

Definition. Let v be a probability measure on C, and let s G [0, oo). 

1. A v-s-supergale is a function d : {0, 1}* — ► [0, oo) that satisfies the condition 

d(w)v(w) s > d(w0)v(w0) s + d(wl)v(wl) s (3.1) 

for all w € {0,1}*. 

2. A v-s-gale is a z^-s-supergale that satisfies (|3.1| ) with equality for all w £ {0, 1}*. 

3. A v-supermartingale is a z^-l-supergale. 

4. A v-martingale is a z^-l-gale. 

5. An s-supergale is a /U-s-supergale. 

6. An s-gale is a /i-s-gale. 

7. A supermartingale is a 1-supergale. 

8. A martingale is a 1-gale. 

Observations 3.1. 1. A subprobability supermeasure on C is a 0-supergale d with d(X) < 1. 

2. A subprobability measure on C is a 0-gale d with d(X) < 1. 

3. A probability measure on C is a 0-gale d with d(X) = 1. 

The following obvious but useful observation shows how gales and supergales are affected by 
variation of the parameter s. 

Observation 3.2. Let v be a probability measure on C, let s, s' £ [0, oo), and let d,d' : {0, 1}* — > 
[0, oo). Assume that 

d{w)u{w) s = d'{w)v{w) s ' 

for allw £ {0,1}*. 

1. d is a v-s-supergale if and only if d' is a v-s' ' -supergale. 

2. d is a v-s-gale if and only if d' is a v-s' -gale. 

For example, Observation implies that a function d : {0, 1}* — > [0, oo) is an s-gale if and 
only if the function d' : {0, 1}* -> [0, oo) defined by d'(w) = 2^- s ^ w \d(w) is a martingale. 
Our next lemma is a generalization of Kraft's inequality ||. 
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Lemma 3.3. Let d be a v-s- super gale, where v is a probability measure on C and s G [0, oo). Then 
for all w G {0, 1}* and all prefix sets B C {0, 1}*, 

d{wu)u{wu) s < d(w)u(w) s . 

ueB 

Proof. We first use induction on n to show that the lemma holds for all prefix sets B C {0, l}- ra . 
For n = this is trivial. Assume that it holds for n, and let A C {0, l}^ n+1 be a prefix set. Define 
the set 

A' = {u G {0, 1}"|u0 eAordG A}, 
and note that A< n is disjoint from A'. Note also that the set 

B = A< n U A' 

is a prefix set. For all w G {0, 1}*, we have 

d(wu)v(wu) s < ^ [d(tt;it0)z/(u)«0) s + d(ural)^(uml) s ] 
«eA =n+ i ueA' 

< d{wu)u{wu) s . 

ueA' 

It follows by the induction hypothesis that for all u> G {0, 1}*, 



wu)v(wu) s 



d(wu)u{wu) s = d{wu)v{wu) s + y^ d( 

< d{wu)v{wu) s + y^ d(wu)v(wu) s 

ueA< n u£A' 

= d(wu)v(wu) s 
ueB 

< d{w)u(w) s . 

This completes the proof that for all n G N the lemma holds for all prefix sets B C {0, 1}- . 

To complete the proof of the lemma, let B be an arbitrary prefix set. Then for all w G {0, 1}*, 

d{wu)u{wu) s = sup y^ d{wu)u{wu) s < d{w)v{w) s . 

ueB neN «e-B<„ 

□ 

Definition. Let d be a z/-s-supergale, where v is a probability measure on C and s G [0, oo). 

1. We say that d succeeds on a sequence S G C if limsupjj^oQ d(S'[0..n — 1]) = oo. 

2. The success set of <i is S 100 ^ = {S G C | <i succeeds on S}. 

We now show how to use the success sets of gales and supergales to define Hausdorff dimension. 
Notation. Let ICC. 

1. G(X) is the set of all s G [0, oo) such that there is an s-gale d for which X C S°°[d]. 
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2. G(X) is the set of all s G [0, oo) such that there is an s-supergale d for which X C 
Note that s' > s G £(Jf) implies that s' E £(X). 

It was shown in p!| that the following definition is equivalent to the classical definition of 
Hausdorff dimension in C. 

Definition. The Hausdorff dimension of a set X C C is dimn(X) = inf £/(X). 

The following trivial fact shows that we could equivalently use Q{X) in place of Q{X) in the 
above definition. 

Observation 3.4. For all ICC, Q(X) = G(X). 

Martin-L6f 's definition of randomness [^] was reformulated in terms of martingales by Schnorr 
p6[ as follows. 

Terminology. A ^-s-supergale is constructive if it is lower semicomputable. 
Definition. Let v be a probability measure on C, and let X C C. 

1. X has constructive v -measure 0, and we write f C onstr(AT) = 0, if there is a constructive f- 
martingale d such that X C S°° [d] . 

2. X has constructive u-measure 1, and we write f CO nstr(X) = 1, if f C onstr(C — X) = 0. 

Definition. If v is a probability measure on C, then a sequence i? € C is v-random, and we write 
i? G RAND^, if the singleton set {R} does not have constructive z^-measure (i.e., there is no 
constructive i/-martingale that succeeds on R). 

It is well-known (and easy to see) that f cons t r (RAND !/ ) = 1. The following known result shows 
that constructive supermartingales can equivalently be used in place of constructive martingales in 
defining randomness. 

Theorem 3.5. (Schnorr (3^, [I?]]) For every computable probability measure v on C and every 
constructive v-supermartingale d there is a constructive v-martingale d' such that S°°[d\ C S°°[d']. 

If v is fj,, the uniform probability measure on C, then we generally omit it from the above 
terminology. A sequence R is thus random, and we write R G RAND, if {R} does not have 
constructive measure 0. 

Optimal constructive supergales are as crucial to constructive dimension as optimal constructive 
supermartingales are to the theory of randomness. Recall the subprobability supermeasure M of 



Theorem 2.1 



Notation. For each s G [0, oo) and w G {0, 1}*, 

d( s '(™) = 2 S HMH. 

Theorem 3.6. For every computable real number s G [0, oo), the function is an optimal 
constructive s-supergale. 
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Proof. Let s G [0, oo) be computable. It is clear from its definition that d^ is a constructive 
s-supergale. To see that d^ has the desired optimality property, let d be an arbitrary constructive 
s-supergale. Fix < a G Q such that ad(A) < 1, and define v : {0, 1}* — > [0, 1] by 

i/(u;) = 2~ s l w lad(u;) 

for all w € {0, 1}*. By Observation |3.2j , is a 0-supergale. Since v(X) = ad(X) < 1, it follows by 
Observation |3,1| that ^ is a subprobability supermeasure on C. Since f is clearly constructive, it 
follows by the optimality of M that there exists a > such that for all w G {0, 1}* , M(w) > ai/(w), 
whence 

d^H = 2 s HM(u>) > 2'M aI/ ( u; ) = aad{w). 
Since aa > this shows that d( s ) is an optimal constructive s-supergale. □ 

We now constructivize the above definition of Hausdorff dimension and develop some funda- 
mental properties of the resulting constructive dimension. 

Notation. Let ICC. 

1- ^constr(^) is the set of all s G [0, oo) such that there is a constructive s-gale d for which 
XQS°°[d]. 

2. ^constr(-X^) is the set of all s G [0, oo) such that there is constructive s-supergale d for which 
XCS°°[d]. 

Note that if s, s 1 ax6 computable real numbers with s f ^> 5, then s G ^/constr 

(X) implies s' G 

^constr(^), and s G Gcoasbt(X) implies s' G Gconstr(X). 

We have seen that gales and supergales can be used interchangeably in defining classical Haus- 
dorff dimension (Observation |3.4[ ) and that constructive martingales and constructive supermartin- 
gales can be used interchangeably in defining randomness (Theorem |3.5| ). In contrast, at the time 
of this writing, we do not know whether constructive gales and constructive supergales can be used 
interchangeably in defining constructive dimension. We also do not know whether an analog of 
Theorem 3.6 holds for constructive s-gales when s < 1. We thus define constructive dimension in 
terms of constructive supergales. 

Definition. The constructive dimension of a set X C C is cdim(X) = inf ^ cons t r (X). 

The following observations are clear. 
Observations 3.7. 1. For all X C Y C C, edim(X) < cdim(Y). 

2. For all ICC, cdim(X) > dim H (X). 

3. cdim(C) = 1. 

4- For all X Q C, if cdim(X) < 1, then /J, C onstr(X) = 0. 

4 Dimensions of Individual Sequences 

The dimension of an individual sequence S G C is simply the constructive dimension of the singleton 
set {S}. 
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Definition. The dimension of a sequence S G C is 



dim(5) = cdim({S}). 

The following theorem, which has no analog either in classical Hausdorff dimension or in the 



resource-bounded dimension developed in | 25| , says that the constructive dimension of a set of 



sequences is completely determined by the dimensions of the individual sequences in the set. 
Theorem 4.1. For all X C C, 

cdim(X) = sup dim(S'). 

sex 



Proof. Let X C C, and let s* — sup se ;sf dim(S') . It is clear by part 1 of Observation |3.7| that 
cdim(X) > s*. To see that cdim(X) < s*, let s be a rational number such that s > s* . It suffices 
to show that cdim(X) < s. 

Since s > s*, for each S G X there is a constructive s-supergale ds that succeeds on S. By 
Theorem |J, then, we have S E S°°[d s } C 5°°[d( s )] for all S € X, whence X C 5°°[d( s )]. Since 



is a constructive s-supergale, this shows that cdim(X) < s. □ 



Hitchcock [14] has recently proven a correspondence principle for constructive dimension. This 
principle says that for any set X C C that is a union of sets (a condition that is certainly 
satisfied if X is E^), the constructive dimension of X is precisely its classical Hausdorff dimension. 



He also noted that this principle, together with Theorem 4.1, implies that the classical Hausdorff 



dimension of every set X C C that is a union of sets has the pointwise characterization 

dimn(X) = sup dim(5). 

s&x 



Theorem 4.1 immediately implies that constructive dimension has the following countable sta- 



bility property, which is also a property of classical Hausdorff dimension. 

Corollary 4.2. For all X ,X 1 ,X 2 , ,..CC, 

/ oo \ 
cdim \X k \ = supcdim(Xfc). 
U=o / fceN 

Our next objective is to prove a dimension reduction theorem that enables us to exhibit se- 
quences of arbitrary A^-computable dimensions in [0,1]. 

Define an approximator of a real number a € [0, 1] to be an ordered pair (a, b) of computable 
functions a, b : N — > Z + with the following properties. 

(i) For all n G N, a{n) < b(n). 



(ii) lim^oo j$ = a. 



It is well known and easy to see that a real number a G [0, 1] has an approximator if and only if it 
is Ag-computable. Moreover, every Arj-computable real number has an approximator (a, b) that is 
nice in the sense that if we let b(k) = Sn=oK n )> then b(k) = o(b(k)) as k — > oo. 

Given an approximator (a, 6) of a Ag-computable real number a G [0,1], we define the (a,b)- 
dilution function 

9(a,b) '■ C — > C 
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as follows. Given S G C, if we write 

S = w wiw 2 ■ ■■ , 

where \w n \ = a(n) for each n G N, then 

9^) = ^-^^-^.... 
Note that g^ b )(S) =1 S for all S G C. 

Theorem 4.3. Let a G [0,1] 6e A°- computable, and let (a, 6) 6e a nice approximator of a. Then 
for allS G C, 

dim(5(( ai6 )(S')) = a • dim(S). 

Proof. We first introduce some notation that will simplify the proof. Let (a, 6) be a nice approxi- 
mator of a. For each G N, let 

fc— l fc-i 
S(£:)5>(n), fe(fe) = ^6(n), 

n=0 n=0 

and note that 

lim — — = a. 

k ^°° b(k) 

In addition to the dilution function g^ a ^) '■ C — > C, we use the function 5 : {0, 1}* — > {0, 1}* defined 
recursively as follows. First, g(X) = A. Next, if w = w'u, where \w'\ = a(k) and < \u\ < a(k), then 
g(w) = g(w')u. Finally, if w = w'u, where \w'\ = a(k) and \u\ = a(k), then g(w) = g(w')uO b ^~ a ^ k \ 
Note that for all S G C, g^ a> b){S) is the unique T G C such that g(w) C T for all uiCS. Note also 
that the function g is one-to-one, so that the string g~ 1 {y) is well defined for each y G range(y). 

Now fix S G C and let = dim(S'). Our objective is to show that dim(gt ab \(S)) = a(3. 

To see that d\m{g^ ab ^(S)) < a(3, let s > and t > a be such that 2 s and 2* are rational. It 
suffices to show that &mi(g(S)) < st. 

Since s > [3, there is a constructive s-supergale ds that succeeds on S. Define a function 
d : {0, 1}* — > [0, 00) as follows. Let y G {0, 1}*. If there does not exist T G C such that y C g( a ,b){T), 
then d(y) = 0. Otherwise, let w be the shortest string such that y C <7(w). Then 

= 2 st M- s H < k( ti; ). 

It is routine to check that d is an st-supergale, and it is clear that d is constructive. Also, for each 
^G{0,1}*, 

d(g(w)) = 2 st ^ w ^- s ^d s (w). (4.1) 
Let e = and fix kg G N such that for all k > ko, 

a(k) 



b(k) 



<t-2e and b(k) < eb(k). 



(Such ko exists because converges to a and the approximator (a, b) is nice.) For all w G {0, 1}*, 
if we choose /c and r such that = a(k) + r and < r < a(/c), and if > ko, then we have 

st\g(w)\ — s\w\ = st(b(k) + r) — s(a(k) + r) 

= st6(fc) — sa(fc) - s(l - i)r 

> stfe(/c) - s(t - 2e)b{k) - seb(k) 

= eb(k). 
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Since e > and S E S°°[<is], it follows by (41) that g( a ,b)(S) € S , °°[d]. Since d is a constructive st- 
supergale, this establishes that dim(g^ a ^(S)) < st, concluding the proof that dim(gr a y\(S)) < a/3. 

To see that dim(g( ab ) (5)) > a/3, let s < a/3 be such that 2 s is rational, and let d be a constructive 
s-supergale. It suffices to show that g( a ,b)(S) ^ S' 00 [(i]. 

Define a function d' : {0, 1}* — > [0, oo) by 

d'H = 2 s ^l- s l 9 ^ld( 5 (u;)). 



Using Lemma 3.3, it is easy to check that d! is a constructive s-supergale. Since s < a/3, we can 



choose t < (3 such that s < at and 2* is rational. The function d" : {0, 1}* — > [0, oo) defined by 

d"(w) = 2^- s ^d'{w) 

is then a constructive t-supergale by Observation [T^. Since t < (3 = dim(S'), it follows that there 
is a constant c € N such that for all w C S", d"{w) < 2 C . 

Let e = |(a — §), noting that this is positive because s < at. Fix ko £ N such that for all 

fc > /CO; 

> £±£ an d 6(fc) < e &(jfe). 
6(fc) ~ * 

(Such ko exists because converges to a, ^j 1 < a, and the approximator (a,b) is nice.) Every 

V != 9(a,b)(S) can be written in the form y = g(w)u, where w Q S, \w\ = a(k), \g(w)\ = b(k), and 
[it | < b(k). For such y we have 

d(y) < 2 s ^d(g(w))<2 b ^d(g(w)) 

= 2 b ^ +s \^ w )\- t \ w \d" (w) 
= 2 b ( k *> +sb '( k y t a( k ) J' (w) 

< 2b(k)+sb(k)-ta(k)+c 

If 1 2/ 1 — b(ko), so that k > &o, then we have 

6(Jb) + s6(fc) - ta{k) < ebk + sb{k) - (a + e)6(ife) = 0, 

whence 

d(y) < 2 C . 

Since this holds for all sufficiently long prefixes y C g( a ,b)(S), it follows that g( a ,b){S) $ S°°[d], 
concluding the proof that dim(g( a 6 )(S)) > a/3. □ 

Notation. For each a € [0, 1], let 

DIM a = {S eC\ dim(S) = a}, 
DIM< Q = {5 e C| dim(,S) < a}, 
DIM <Q = {S e C| dim(S) < a}. 

Observation 4.4. RAND C DIMi. 

Proof. This follows immediately from part 4 of Observation |3.7|. □ 
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An important result in the theory of random sequences is the existence of random sequences 



in A[j. We now use this fact and Theorem L3 to show that there are A!] sequences of every 
Ag-computable dimension in [0,1]. 

Theorem 4.5. For every A®- computable real number a € [0, 1], DIM a n A!] 7^ 0, i.e., there is a 
A® sequence S such that dim(5) = a. 

Proof. Let a € [0, 1] be A^-computable. It is well known and easy to see that a has an approximator 
(indeed, this characterizes A^-computability), and it is routine to transform an approximator of a 



into a nice approximator (a, 6) of a. It is well known (see [49, 30 [ or |]22|) that there is a sequence 



R G RAND nA§. Let S = g^ b) {R). Then Theorem [Dj| and Observation £D| tell us that 

dim(5) = adim(R) = a. 

□ 

Three remarks on the proof of Theorem [4.5| should be made here. First, the proof that RAND n 
Ag 7^ using Kreisel's Basis Lemma fl9| , |49], |50|, [30|[ and the fact that RAND is a £!] set cannot 
directly be adapted to proving that DIMq, nA^/i because Terwijn @ has shown that DIM a 
is not a set. Second, Mayordomo |28|| has recently generalized Chaitin's Q construction || to 
give an alternative construction of sequences in DIM a n A§. Third, our proof of Theorem 4.5 



via 



Theorem yields even more, namely, that if a, j3 S [0, 1] are A^-computable with a > /3, then 
every sequence in DIM Q is 1-equivalent to some sequence in DIM^. 



The following theorem shows that Theorem L5 cannot be improved to T,® or sequences. 
Theorem 4.6. Sju^C DIM . 

Proof. Let 5 £ Sj. By symmetry, it suffices to show that dim(5) = 0. For this, let < s S Q. It 
suffices to show that dim(5) < s. 



By standard techniques |3T], 42|, let Sq, Si, . . . be a sequence of elements of C with the following 
properties. 

(i) For each t, St contains only finitely many l's. 

(ii) For each t and n, St[n] < St+i[n]. 

(iii) For each n, S[n] = lim^oo St [re]. 

(iv) The set {(t, n)\St [n] = 1} is computably enumerable. 

That is, St is the "i th finite approximation of S 1 ." 

Define a function d : {0,1}* — > [0, 00) as follows. First, d(A) = 1. Next, assume that d(w) has 
been defined, where \w\ = (2) for some integer n > 1. For each u G {0, 1}™, define 



d(wu) 



^ if(3t)s t [®..r 2 i )-i]=u 



I otherwise, 
noting that \wu\ = (2) + n = (^2 )• "^ or eac ' 1 u suc h that < \u\ < n define 



d(wu) = 2-^-^ d{wuv). 

\v\=n—\u\ 



Since there are at most n + 1 strings u € {0, l} n for which d{wu) > 0, it is clear that d is an s- 
supergale. It is also clear that d is constructive and that d succeeds on S, whence dim(S') < s. □ 
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The rest of this section concerns the constructive dimensions of the dimension classes DIM< Q 
and DIM <a . We first note that for every a G [0,1], DIM< a is the largest set of constructive 
dimension a. 

Theorem 4.7. For every a S [0, 1], the set DIM< Q has the following two properties. 

1. cdim(DIM< a ) = a. 

2. For all ICC, if cdim(X) < a, then X C DLM< a . 

Proof. Part 1 follows immediately from Theorem [4.1| , Theorem 4.5, and the fact that the A!]- 
computable reals are dense in R. Part 2 follows immediately from part 1 of Observation 3.7. □ 



Part 1 of Theorem 47 has the following immediate consequence. 
Corollary 4.8. For every a € [0, 1], 

cdim(DIM< Q ) = a. 

We show in section 6 below that 

cdim(DIM Q ) = a 

for all reals a 6 [0,1]. 



5 Dimensions of Individual Strings 

In the preceding two sections we have constructivized classical Hausdorff dimension and thereby 
defined the dimensions of individual infinite binary sequences. We now push this one step further 
by constructivizing and discretizing classical Hausdorff dimension in order to define the dimensions 
of individual finite binary strings. 

Recall that the dimension of a sequence S is the infimum of all s > for which there exists 
a constructive s-supergale d such that the values of d(S[0..n — 1]) are unbounded as n — > oo. To 
define the dimensions of finite strings, we modify this definition in three ways. 

I. We replace supergales by termgales, which are supergale-like constructs with special require- 
ments for handling the terminations of strings. 

II. We replace "unbounded as n — » oo" by a finite threshold. 

III. We make the definition universal by using an optimal constructive termgale. 

We now carry out this development. 

Supergales are well suited to defining the dimensions of infinite sequences, but an adequate 
definition of the dimensions of finite strings must also involve betting on the point at which a given 
string terminates. We use the termination symbol □ to mark the end of a binary string. We work 
in the set 

T = {0,1}* U{0, 1}*D, 

consisting of all terminated binary strings (elements of {0, 1}*D) and prefixes thereof. The following 
definition is the main idea of this section. 
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Definition. For s G [0, oo), an s-termgale is a function 

d : T -> [0, oo) 
such that d(X) < 1 and for all w G {0, 1}*, 

d(w) > 2~ s [d(w0) + d(wl) + d(wD)}. (5.1) 

An s-termgale d is a strategy for betting on the successive bits of a binary string and also on 
the point at which the string terminates. We require the initial capital d(X) to be at most 1. When 
d is used to bet on a string w, the final capital is d(wU). 

The payoff condition (|5.1[) may at first glance seem suspicious. In the case s = 1, this says that 



d( W Q) + d( W l) + d( W D) 
d(tw) > (5.2) 

for all w G {0,1}*. If each of 0, 1, and □ is equally likely to occur, independently of all prior 
bits, then (ET^) implies that the conditional expected capital after a bet, given that w has occurred 
before the bet, is 

d(wO) + d(wl) + d(wD) _ 2 

3 " 3 {Wh 

whence the payoffs are much less than fair, even if equality holds in ( |5.2| ). However, the assumption 
that 0, 1, and □ are equally likely to occur is not reasonable because it forces strings to be very short 
with overwhelming probability. (In fact, this assumption implies that the average string is only 
two bits long. In contrast, the average length of a string with respect to the optimal constructive 
subprobability measure m is infinite.) Since we want our theory to apply to long strings, the 
termination symbol □ should be regarded as having a vanishingly small probability. 

The 1-termgale payoff condition ( f^% ) is exactly the supermartingale (i.e., 1-supergale) payoff 
condition with the additional requirement that the 1-termgale must without compensation divert 
some of its capital to bet on □, i.e., the possibility that there is no next bit. Since □ can only occur 
once, the overall impact of this requirement is modest. However, the impact is real, and we shall 
see that it is exactly what is needed. 

Example 5.1. Define d : T — > [0, oo) by the recursion 

d(X) = 1, 

d(w0) = §d(tfl), 

d(wl) = d{wD) = \d{w). 

It is clear that d is a 1-termgale. If w is a binary string of length n with uq 0's and n\ l's, then 

d(wD) = (|) no (|) ni+1 

_ 2 n o(l+log3)-2(n+l) 

In particular, if uq is significantly larger than 1+1 Q g3 (n + 1) ~ 0.7737(n + 1), then d(wO) is signifi- 
cantly greater than d(X) even though d loses three-fourths of its capital when the □ appears. 



The following analog of Observation |3.2j is obvious but useful. 
Observation 5.2. Let d,d' : T —> [0, oo) and s, s' G [0, oo). If 

2" s N(f(x) = 2- s '\ x \d'(x) 
for all x G T, then d is an s-termgale if and only if d' is an s' -termgale. 
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In particular, if d is a O-termgale and s G [0, oo), then the function d! defined by 

d'{x) = 2 s \ x \d(x) 

for all x G T is an s-termgale, and every s-termgale can be obtained from a O-termgale in this way. 
Lemma 5.3. If s G [0, oo) and d is an s-termgale, then for all u G {0, 1}*, 

2~ s \ w \d(uwn) < 2 s d(u). (5.3) 

we{o,i}* 

Proof. For the first part of the proof, assume that d is a O-termgale, and let u G {0, 1}*. We begin 
by using induction on m to show that 

^2 d{uwD) + ^ d{uw) < d(u) (5.4) 
w&{o,i} <m we{o,i} m 

for all m G N. For m = 0, this is trivial. Assume that it holds for m. Then 

S2 d{uwU) + ^2 d(uw) 
«;e{o,i}< m + 1 toe{o,i} m + 1 

= ^2 d(uwD) + ^2 d(uwU) + ^2 [d(uw0) + d(uwl)] 

w£{0,l}< m we{0,l} m toG{0,l} m 

< ^ d(uwD) + ^2 d(uw) 

< d{u) 

by the induction hypothesis. This confirms that ([x4|) holds for all m G N. It follows immediately 
that 

^2 d{uwU) < d(u) 

«)G{0,l}< m 

for all m G N, whence 

^2 d{uwU) < d(u). 
we{o,i}* 

This is the case s = of d5T3|). 

Now assume that d is an s-termgale, where s G [0, oo) is arbitrary. Define d' : T — > [0, oo) by 
rf'(x) = 2" s l a; l(i(x) for all x G T. Then d' is a 0-termgale by Observation ^2, so the first part of 
this proof tells us that for all u G {0, 1}*, 

Y 2- s \ w \d(uwn) = 2 S I" D I ^2 d'{uwU) 
wg{o,i}* «ie{o,i}* 

< 2 s \ uQ \d'{u) 

= 2 s d{u). 

□ 

To define optimal termgales we need uniformity in the parameter s. 

Definition. 1. A termgale is a family d = {S- S '\s G [0, oo)} such that each d^ is an s-termgale 
and for all s, s' G [0, oo) and x G T, 

2- s l x, ld (s) (x) = 2" s 'l a; ld( s ')(a;). 
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2. A termgale d is constructive if is constructive. 

Definition. A constructive termgale d is optimal if for every constructive termgale d there is a 
constant a > such that for all s G [0, oo) and u) G {0, 1}*, > ad^ (wD). 

Definition. The termgale induced by a subprobability measure p on {0, 1}* is the family d\p] = 
{d\p]( s >\s G [0,oo)}, where each d[p]( s ) : T — > [0, oo) is defined by 

d[p} {s \x) = 2 S ^ P{w) 
iue{o,i}* 

for all x G T. 

Theorem 5.4. If p is an optimal constructive subprobability measure on {0,1}*, then d[p] is an 
optimal constructive termgale. 

Proof. Assume the hypothesis. It is clear that d[p] is a constructive termgale. To see that d[p] is 
optimal, let d = {d^\s G [0, oo)} be an arbitrary constructive termgale. Define p : {0, 1}* — > [0, oo) 
by p(w) = d(°)(u>D) for all w G {0, 1}*. By Lemma ^ (with u = A), p is a subprobability measure 
on {0, 1}*, and p is constructive because d is constructive. It follows by the optimality of p that 
there exists a > such that p(w) > ap(w) for all w G {0,1}*. Then for all s G [0, oo) and 

d[p]^( w a) = 2 s ^p( w ) 

> 2 s \ wn \ap(w) 
= 2 s ^ad^{wD) 
= ad^(wD), 

so d[p] is optimal. □ 
Corollary 5.5. There exists an optimal constructive termgale. 



Proof. This follows immediately from Theorems 5.4 and 2.1. □ 



We can now implement the ideas I, II, and III described at the beginning of this section. 

Definition. If d is a termgale, I G Z + , and w G {0, 1}*, then the dimension of w relative to d at 
significance level I is 

dim^(u>) = inf{s G [0, oo) | d (s) (u>D) > I}. 
We write dim^u;) for dim^(w). 

Theorem 5.6. If d is an optimal constructive termgale, then for every constructive termgale d 
and every I G Z + , there is a constant 7 G [0, 00) such that for all w G {0, 1}* ; 

dinirfw) < dim.d(w) H ; — , . (5-5) 

a 1 + \w\ 
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Proof. Let d be an optimal constructive termgale, let d be an arbitrary constructive termgale, and 
let I E Z + . By the optimality of d, there is a constant a G (0,1] such that for all s G [0, oo) and 
w E {0,1}*, #•)(«;□) > adW («;□). Let 7 = log /-log a, and note that 7 G [0,oo). Let 10 G {0,1}* 
be arbitrary. To see that (5.5) holds, let s > dim^(w) + 1+ y . It suffices to show that dS s \w\J) > I. 



Let s\ = s — 1+ T 1 . Then > dim^(t(;), so 



= a2^d {si) (wn) 
> a2 7 
= I. 



□ 



Corollary 5.7. // d\ and d% are optimal constructive termgales and li,fo G Z + ; then there is a 
constant a G [0, 00) such that for all w G {0, 1}*, 



dim'j (w) — dim 1 ? (w) 



a 

< 



1 + \w\ 



Corollary 5.7 says that if we base our definition of dimension on an optimal constructive termgale 



d, then both the particular choice of d and the choice of a significance level I have negligible 
impact on the dimension dimj(w). We thus fix an optimal constructive termgale d\j and define the 
dimensions of finite strings as follows. 

Definition. The dimension of a string w G {0, 1}* is 

dim(io) = dimd n (w). 

We have seen that the dimension of a sequence is at most 1. In contrast, we will see in section 
6 that the dimension of a string may exceed 1. However, regardless of our choice of dpj, there is an 
upper bound on the dimension of strings. 

Lemma 5.8. There is a constant c G N such that for all w G {0, 1}*, dim(w) < c. 
Proof. For each s G [0, 00), define d^ : T — > [0, 00) by 



' 2 (s-2)M if xG {o,l}* 
2 (s-2)\x\+i if a; G {o,l}*D, 



d {s \x) 

and let d = {d^\s G [0, 00)}. It is easy to see that d is a constructive termgale and d^(w\3) = 2 



for all w G {0, 1}*, whence dim^(w) < 2 for all w G {0, 1}*. It follows by Theorem 5J3 that there is 
a constant 7 G [0, 00) such that for all w G {0, 1}*, 

dim(w) < 2 H ^— < 2 + 7. 

1 + |ui| 

Thus the present lemma holds with c = 2 + [7] . □ 

We conclude this section by characterizing the dimension of a sequence in terms of the dimen- 
sions of its finite prefixes. 
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Theorem 5.9. For all S G C, 

dim(S') = liminf dim(5[0..n — 1]). 

n^oo 

Proof. Let S G C. To see that dim(S) < liminf dim(5[0..n — 1]), let s and s' be rational 
numbers such that s' > s > liminfn^oo dim(5[0..n — 1]). It suffices to show that dim(5) < s'. By 
our choice of s, there is an infinite set J C N such that for all n G J, dim(<5[0..n — 1]) < s, whence 
d^ } (5[0..n - 1]D) > 1. Define d! : {0, 1}* -» [0, oo) by = dg #) (uO for all «; G {0, 1}*. Then d' 

is a constructive s'-supergale and for all n G J, 

d'(5[0..n- 1]) = d^O^Cra-l]) 

= 2( s '- s ) n dg ) (5[0..n- 1]) 

> 2( a '- 8 ) n 2- a df?(S[0..n-lp) 

^ 2( s ' — s ) n—s 

Since J is infinite, this implies that S G 5°°[d'], whence dim(5) < s'. 

To see that dim(S) > liminf dim(5[0..n — I]), let s' and s" be rational numbers such 
that s' > s" > dim(jS'). It suffices to show that there exist infinitely many n G N for which 
dim(iS'[0..n— 1]) < s'. Since s" > dim(£), there is a constructive s"-supergale d such that S G S°°[d]. 
Define d' : T -> [0, oo) by 




d(x) if x G {0, 1}* 

(2 s ' - 2 s ")d{w) if x = wD G {0,1}*D. 



Then d! is a constructive s'-termgale, so if for each s G [0, oo) we define d^ : T —> [0, oo) by 
d^ s \x) = 2^ s ~ s )\ x \d'(x), then the family d = {d^\s G [0, oo)} is a constructive termgale. It 
follows by the optimality of d^ that there is a constant a > such that for all s G [0, oo) and 
w G {0,1}*, dg } (u>D) > adW («;□). Since S G there are infinitely many n G N such that 

a(2 s ' — 2 s ")d(S'[0..n — I]) > 1. For all such n we have 

d { n\s[0..n - 1}D) > a$ s '\S[0..n- 1]D) 

= ad'(S[0..n - 1]D) 

= a(2 s ' - 2 s ")d(,S[0..n - 1]) 

> 1, 

whence dim(5"[0..n - 1]) < s' . □ 



6 Dimension and Kolmogorov Complexity 

In this section we show that the Kolmogorov complexity of a string is (up to an additive con- 
stant) the product of its length and its dimension. We use this to derive a new proof of a recent 
characterization of the dimension of a sequence in terms of the Kolmogorov complexities of its 
prefixes. This latter result is used to establish the existence of sequences of all dimensions in [0,1]. 
We also review some previous work on martingales, supermartingales, Kolmogorov complexity, and 
Hausdorff dimension. 
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Theorem 6.1. There is a constant c G N such that for all w G {0, 



K(w) — \w\ dim(w) 



< c. 



Proof. Let m be the optimal subprobability measure on {0, 1}* defined in section 2. The key fact 
is that for all w G {0, 1}* and s G [0, oo), 

d[m] {s \wO) > 1 2 s l wD lmH > 1 

1 , 1 

s > ; — t log 



so 



This implies that 



1 + m(w) ' 



dim d[m] ( w ) = — 

11 1 + \w\ m(w) 



log 



1 



log- 



m(w) 



(1 + \w\)dim d[m] (w). 



(6.1) 



To complete the proof, fix constants cq, c\,C2 E N such that for all w G {0, 1}* 

1 



K{w) — log 



m(w) 



< co, 



(6.2) 



dim dfml(^) - dim(w)| < 



ci 



1 + ku 



(6.3) 



and 



dim(u;) < c 2 . 



(6.4) 



(The constants cq and C2 exist by Theorem 2.2 and Lemma 5.8, respectively. The constant c\ exists 



by Theorem 5A and Corollary 5/7.) Let c = cq + c\ + C2- Then for all w G {0, 1}*, (|6.1| ) and ( |6.3| 
tell us that 



log 



1 



m(w) 



(1 + \w\) dim(w) 



< ci, 



and ( |6.4| ) tells us that 

|(1 + |u;|) dim(w) — \w\ dim(u>)| < C2- 
By (|6.2|) , (|6.5|) , (|6^), and the triangle inequality, we have 

\K(w) — \w\ dim(w)\ < c 

for all w G {0,1}*. 



(6.5) 



(6.6) 



□ 



In addition to giving a new characterization of Kolmogorov complexity, Theorem p,l| enables us 
to derive bounds on dimension from known bounds on Kolmogorov complexity. For example, we 
have the following. (Note: dim(|u>|) is dim(z), where z is the \w\ th string in a standard enumeration 
of {0,1}*.) 
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Corollary 6.2. There exist constants c±,C2 G N with the following properties. 
1. For all w G {0, 1}*, d\m(w) < 1 + dim(H) + 

For all n, r G N, i/ we choose w G {0, l} n according to the uniform probability measure on 
{0,1}", then 

T 

&\m.{w) > 1 + dim(|u;| 



Pr 



\w 



> 1 - 2 C2_ 



Proof. This follows immediately from Theorem 6.1 and Theorem 3.3.1 of [22] 
Thus most strings w satisfy 

1 



□ 



dim(w) = 1 + dim(|u)|) ± O I -. — r I . 



w 



It follows readily that dim(w) often exceeds 1. 

Theorem |6.1| characterizes Kolmogorov complexity in terms of dimension. We use this to give 
a new proof of a recent characterization of the dimensions of sequences. 



Corollary 6.3. (Mayordomo 



For all SeC, 



dim(S') = lim inf 



K(S[0..n- 1]) 



n 



Proof. By Theorem 6.1 



lim inf 

n— >oo 



K(S[0..n-l}) 



n 



liminf dim(S'[0..n — 1]), 



so the corollary follows by Theorem |5.9| . □ 
We conclude this section with an easy proof that there exist sequences of all dimensions in [0,1]. 
Theorem 6.4. For every a G [0, 1], DIM a / 0. 



Proof. Let a € [0,1]. If a = or a = 1, then DIM Q ^ by Theorem 4J3 or Observation 
respectively, so assume that a £ (0, 1). Let R G RAND, and let S be the sequence constructed by 
the following nonterminating, noncomputable procedure. 

for n := to oo do 

S[n] := if K(S[0..n - 1]) < an then R[n] else 

Every sequence of the from R' = wR[\w\. .oo) is random and thus satisfies ^K(R'[0..n — 1]) — > 1 as 
n — > oo | |22} |. On the other hand, every sequence of the form T = w0°° satisfies K(T[0..n— 1]) = o(n) 
as n — > oo [22]. Finally, it is well-known that there is a constant c G N such that for all w G {0, 1}* 
and b G N, |-ftT(u>6) — K(w)\ < c. These three things together imply that the sequence S satisfies 

r K(S[0..n-l]) 

lim = a, 



whence S G DIM a by Corollary 6.3. 



□ 
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Corollary 6.5. For every a € [0, 1], 



cdim(DIM a ) = a. 



Proof. This follows immediately from Theorems |4.1| and 6.4 



□ 



We conclude this section by discussing some earlier work relating martingales, supermartingales, 
and Kolmogorov complexity to Hausdorff dimension. Schnorr |37|, 39] defined a martingale d to 
have exponential order on a sequence S if 



logrf(S[0..n-l]) ^ 
lim sup > U 



(6.7) 



and proved that no computable martingale can have exponential order on a Church-stochastic 
sequence. Terwijn |48j has noted that (|6.7D is equivalent to the existence of an s < 1 for which the 
■s-gale d( s \w) = 2^ s ^^^d(w) succeeds on S. Thus, in the terminology of [25], Schnorr's result says 
that the set {5} has computable dimension 1 for every Church-stochastic sequence S. 
Ryabko pi] proved that 



dim H < S 



r K(S[0..n - 1]) 
lim inf ^ < a 



n 



a, 



and Cai and Hartmanis Q proved that 



dirm 



S 



. t K(S[0..n - 1]) 
hm mf — = a 



■n 



a 



(6.8) 



(6.9) 



for all a € [0,1]. In light of Corollary 6.3, ( |6.S|) and ( |6.9| ) say that dimH(DIM< Q ) = a and 
dim H (DIM Q ) — a, so ( |6.8| ) and (|6.9[) can be regarded as classical analogs of Theorem |4.7j (l) and 
Corollary |6.5| , respectively. 
Ryabko [^3f| proved that 



dimn(-X^) < sup < lim inf 



K(S[0..n-l]) 



n 



SeX 



for all X C C, and Staiger [46] established the existence of sets ICC for which 

K(S[0..n-l}) 



dimn(X) < sup < lim sup 



n 



SeX 



(6.10) 



(6.11) 



By Theorem [4.1| and Corollary ( 6.1Cj ) can now be seen as a statement of Observation |3.7| (2). 

Ryabko |^5| and Staiger |47|] defined the exponent of increase of a martingale dona sequence 
S to be the number 



Xd(S) = lim sup 



logd(5[0..n- 1]) 



n 



(6.12) 



which is the left-hand side of ( |6.7| ). (We are using Staiger's notation here.) Both papers paid 
particular attention to the quantity 



\{S) = sup{A^(5)|d is a computable martingale}. 



(6.13) 
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By Terwijn's above-mentioned observation, 1 — A(S) is precisely the computable dimension of {S} 



in the terminology of |25|. Ryabko |55| proved that 



A(S) < 1 - liminf 



K(S[0..n - 1]) 



n 



(6.14) 



for every sequence S. By Corollary |6.3| , we can now regard ( 6.14Q as stating that dim(S) is no 
greater than the computable dimension of {S}. Ryabko |35| also proved that 

dim H ({S|A(S) > a}) = 1 - a (6.15) 



for all a € [0,1]. This is yet another analog of Theorem 4.7(1), saying that for all a £ [0, 1] the set 
DIM^ omp , consisting of all sequences S such that the computable dimension of {S} is at most a, has 
Hausdorff dimension a. (Note: The earlier paper |54[] proved results similar to ( |6.14 ) and ( 6.15| ), 
but with A(S) replaced by a different quantity, which we may call X'(S), in which the algorithm 
for the martingale is only required to halt on inputs of the form w, wO, or wl for prefixes w of S. 
It is easy to see that X'(S) is bounded below by X(S) and above by 1 — dim(S).) 

Staiger [47] provided even more insights. If d = is the optimal constructive supermartingale 
of Theorem KB above, then Staiger 's Ad (SO is exactly 1 — dim(S). He proved that 



dim H (X) 



sup < lim inf 

I n— >oo 



K(S[0..n-l\) 



ii 



SeX 



(6.16) 



for every E° set ICC. In light of Theorem 4.1 and Corollary |6.3] , this is equivalent to the result 
by Hitchcock 14], mentioned in section 4 above, that cdim(X) = dimn(-X^) for every E[] set ICC. 
(It should be noted however, that the Staiger and Hitchcock results both preceded the Mayordomo 
pq| proof of Corollary |6.3| and that Hitchcock's result holds for arbitrary unions of Hj 1 sets.) Staiger 
[47] also proved that 



sup 



MXAS) 



d is a computable martingale > = 1 — dimn(I) 



(6.17) 



for every E^ se ^ X C C It is now easy to see that this is equivalent to the result by Hitchcock ]lj 
that the computable dimension of a E^ set X C C is precisely its Hausdorff dimension. Finally, 
Staiger |^7j characterized Hausdorff dimension in terms of entropy rates, and Staiger [45] gave an 
enjoyable exposition of his and Ryabko's results in terms of an infinite game. 

This brief review does not exhaust the results of the cited papers, but it does indicate the 
emergence of a rich network of relationships among martingales, supermartingales, Kolmogorov 
complexity, Hausdorff dimension, constructive dimension, and computable dimension. 



7 Dimension and Biased Randomness 

We now investigate the dimensions of sequences that are random relative to computable sequences 
of convergent biases. We first recall two known theorems concerning such sequences. 

Given a bias sequence (3 = (/?o, fti, P2, ■ ■ ■), we write RAND^ for the set of all sequences that are 

random relative to the /3-coin-toss probability measure fjP defined in section 2. For each nonempty 
string w G {0, 1} + , let 

#(l,w) 
tieqiw) = — ; — j — , 
\w\ 
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where #(&, w) is the number of times the bit b occurs in w. For each f3 G [0, 1], we define the set 



FREQ. = (s G C lim freq(S[0..n -l])=p\. 

The following well-known theorem is a constructive version of the strong law of large numbers. 

Theorem 7.1. (folklore) If P is a computable sequence of biases that converge to P G [0,1], then 
RAND^ C FREQ^. 

Definition. Two sequences of biases and 0' are square- summably equivalent, and we write (3 « 2 
P', if E£o(A-^i) 2 <oo. 

The next theorem is a constructive version of a classical theorem of Kakutani IT 



Theorem 7.2. (van Lambalgen | 49| , 5C], Vovk |5l[] ) Xei /? anrf /?' 6e computable sequences of biases 
that converge to (3 G (0, 1). 

I // /? « 2 (3', then RAND^ = RAND*,. 
2. If (3^ 2 (3', then RAND^ n RAND^, = 0. 

It is well-known (and easy to see) that a real number is A^-computable if and only if it is 



the limit of a computable sequence of reals. Thus Theorems 7.1 and |7.2j tell us that for each 
Ag-computable bias f3 G (0, 1), the set FREQ^ contains infinitely many disjoint sets of the form 
RAND 3, where /3 is a computable sequences of biases converging to /?. This section is concerned 
with the dimensions of the sequences in these sets RAND^. Our main result uses three lemmas. 

Our first lemma follows immediately from a result in 25], but it is central to our development 
and a direct proof is brief, so we give it here. Recall the notation E^g(a), the binary entropy 
function 7i(/3), and the binary Kullback-Leibler divergence D(/3 \\ a) discussed in section 2. 

Lemma 7.3. For all (3 G [0, 1], cdim(FREQ /3 ) < H{(3). 

Proof. Let j3 G [0,1], and let s be a rational number with s > TL{(3\ It suffices to show that 
cdim(FREQ /3 ) < s. 

Let e = s . Fix a rational number r G (0, 1) such that 

D(fl || r) < e. (7.1) 
Define d : {0, 1}* — > [0, oo) by the recursion 

d(X) = l, 

d(w0) = 2 S (1 -r)d(w), 
d(wl) = 2 s rd(w). 

It is clear that d is a constructive s-gale. 

To see that FREQ^ C S°°[d}, let S G FREQ^. For all n G Z+, let w n = S[0..n - 1] and 
p n = freq(u> n ). Since S G FREQ^, there exists no G Z + such that for all n > n , 

H( Pn )<H(P)+e (7.2) 
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and 



For all n G Z + we have 



so 



log d(w n 



D( Pn || r) <D{(3 || r) + e. 
d(w„) = 2 sn r # ( 1 ' M '")(l - r )*(°> Wn \ 

n[s + jO n log r + (1 - /?„) log(l - r)] 
1" 



(7.3) 



n 



n 



s - E Pn log 



1 Pr, 



,Pn r 

= n[s- TC(p n ) - D(p n || r)) . 
It follows by flTlD , (|7T2|), and (|7j) that for all n > no, 

logd(w n ) > n[s — W(/3) — 3e] = en. 
Thus 5 G 5°°[d]. This shows that FREQ^ C S°°[d], whence cdim(FREQ /3 ) <s. □ 

Our second lemma gives an asymptotic estimate of log u^(S'[0..n — 1]) when (3 converges to 
(3 G (0, 1) and S has limiting frequency f3. 

Lemma 7.4. If (3 is a bias sequence that converges to (3 G (0, 1), then for all S G FREQ^, 

log //(5[0..n - 1]) = -H(/3)n + o(n) 

as n —> oo. 

Proof. Using the abbreviations 

flog(l-A) if5[*]=0 
\ log ft ifS[*] = l, 

_ flog(l-/3) if5[i]=0 
\log(/3) if S[i] = l, 
a n = freq(/S[0..n — 1]), 

the hypothesis tells us that 

n-l 

log/(S[0..n- 1]) = 



i=0 
n-l 

I>+°a)) 



i=0 
/n-l 



n[(l - On) log(l - /3) + a n log /3] + o(n) 

n[(l - /? - o(l)) log(l - /3) + (/? + o(l)) log /?] + o(n) 

-W(/3)n + o(n) 



as n — ► oo. 



□ 
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Our third lemma is the crucial one. Its brief proof uses a natural transformation of an s- 
supergale to a /3-supermartingale. 

Lemma 7.5. If (3 is a computable sequence of biases that converge to [3 G (0, 1), then for every 
computable s G [0,Tt(fl)) and every constructive s-supergale d, the set S°°\d] has constructive (3- 
measure 0. 



Proof. Let (3, /9, s, and d be as given. By Theorem |7. l| , the set FREQ^ has constructive /3-measure 
1, so it suffices to show that the set S^fd] n FREQ^ has constructive /3-measure 0. Let 

a(w) = 2- s ^d(w) 



for all w G {0, 1}*. By Observation a is a 0-supergale. Since d is constructive and s and (3 are 
computable it follows that the function 

d' = ^ 

is a constructive /3-supermartingale. 

Now let S G FREQ^, and for each n G N, let w n = S[0..n — 1]. Since s < H((3), Lemma 7A 
tells us that for sufficiently large n G N, 

sn + log (J,P(w n ) < 0, 

whence 

A<( \ rf ( U 'n) ^ •/ \ 

d (w n ) = j- — - > d[w n ). 

2 sn ^(w n ) 

This shows that S°°[d] n FREQ^ C S°°[d']. Thus d' testifies that 5°° [d] n FREQ^ has constructive 
/3-measure 0. □ 



By Lemma 7J3 every sequence in FREQ^ has dimension at most TL{(3). This upper bound is 
not in general tight. For example, if (3 is A^-computable, it is easy to see that there are sequences 
of dimension in FREQ^. Nevertheless, the following theorem says that the upper bound Ti.(/3) is 
achieved by every sequence in each of the sets RAND^ for which (3 is computable and converges to 
f3. 

Theorem 7.6. If (3 is a computable sequence of biases that converge to (3 G (0, 1) and R G RAND^-, 
then dim(fl) = H((3). 



Proof. Assume the hypothesis. By Theorem 7.1 and Lemma |7.3| , dim(i?) < 7~C(f3). To see that 
dim(i?) > H(/3), let s G [0, H(P)), and let d be a constructive s-supergale. By Lemma 75, 5°°[d] 
has constructive (3- measure 0. Since R G RAND^, this implies that it! G" S 00 ^. Since this holds 
for all s G [0,Tt(f3)) and all constructive s-supergales d, it follows that dim(i?) > Tt(f3). □ 



Note that Observation AA is exactly the case (3 = (i, i, i, . . . ) of Theorem [7(]. Note also that 
Theorem 7.6 can be used to give a second (albeit less informative) proof of Theorem 4.5. 
Besicovitch [JTJ proved that dim H (FREQ<^) = H{(3) for all (3 G [0, |], where 



FREQ^ = { S G C 



limsupfreq(S'[0..n — 1]) < (3 



Good |L2j conjectured that the limit supremum could be replaced by a limit here, thus obtaining 
dim H (FREQ /3 ) = H{(3) for all G [0, 1]. Eggleston [| (see also §, |TTJ] ) proved Good's conjecture. 
The following corollary is a constructive version of Eggleston's theorem. 
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Corollary 7.7. If G [0, 1] is computable, then cdim(FREQ /3 ) = H(J3). 



Proof. If /3 = or /3 = 1, this follows immediately from Lemma |7.3| , so assume that (3 € (0,1) is 
Ag-computable. Then there is a computable bias sequence /3 that converges to (3, so Lemma 7.3 
and Theorems and 7J; tell us that cdim(FREQ^) = 7i{(3). □ 



Computable bias sequences that converge slowly to 5 have played an important role in the 
investigation of stochasticity versus randomness. First, Theorem |7.2] implies that if (3 is a bias 
sequence such that E£o(A ~ W = 00 > then RAND-g> n RAND = 0. Also, van Lambalgen 
[49, 5C] proved that if (3 is any computable bias sequence that converges to 5, then every element 
of RAND-g» is Church-stochastic. Taking (3 to converge to |, but to do so slowly enough that 
YliZoWi ~ \) 2 = 00 ( e -g-) Pi = 5 + /. ,. ), this gave a new proof that not every Church-stochastic 



sequence is random. More significantly, Shen' [41| strengthened van Lambalgen's latter result by 
showing that if (3 is any computable bias sequence that converges to i, then every element of 
RAND^ is Kolmogorov-Loveland stochastic. Again taking j3 to converge to \ slowly enough 
that YHS=o(Pi ~ \) 2 = 00 > ^ s allowed Shen' to conclude that not every Kolmogorov-Loveland 
stochastic sequence is random, thereby solving a twenty-year-old problem of Kolmogorov |l6|, [l~8| 
and Loveland [23|, 24]. Theorems |7.6| and 7.2 have the following immediate consequence concerning 



such sequences f3 . 

Corollary 7.8. If (3 is a computable sequence of biases that converge to | slowly enough that 

E~o(A - \) 2 = °°> then 



RAND-g> C DIMi - RAND. 

That is, every sequence that is random with respect to such a bias sequence (3 is an example 
of a sequence that has dimension 1 but is not random. 
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